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Introduction 


Effective international S&E partnerships advance the S&E enterprise and energize U.S. 
innovation and economic competitiveness, but they also have great potential to improve 
relations among countries and regions and to build greater S&E capacity around the world. 1 

Over a decade ago, the National Science Board (NSB) highlighted the importance of international 
collaboration in its call for increased government commitment to promoting international science 
and engineering (S&E) research and education. 2 The NSB also identified the National Science 
Foundation (NSF) as having an important leadership role in international S&E research and 
education activities. 3 A specific area of focus for NSF was promoting "...increased participation in 
international S&E activities by younger U.S. scientists and engineers from diverse backgrounds, 
especially those in the early stage of their careers, in order to develop an internationally competitive 
and globally-engaged S&E workforce." 4 

Among NSF's postdoctoral programs, the International Research Fellowship Program (IRFP) is unique 
in its emphasis on providing postdoctoral fellows with international research experiences. 5 
Established in 1992, IRFP provides financial support to postdoctoral scientists for a research 
experience abroad lasting from 9 to 24 months; 6 there is no restriction on the geographical location 
of the hosting institution. IRFP aligns closely with the NSB's call for NSF to support the international 
science engagement of scientists and engineers. 

NSF contracted with Abt Associates to conduct an evaluation of IRFP, to gather evidence about 
whether the IRFP program has achieved its goal of furthering the collaborative activities and 
international partnerships of early career STEM researchers. The evaluation found evidence that the 
IRFP program is meeting its goals, which are to: 

• Introduce early career scientists and engineers to opportunities for international research 
collaboration; 

• Build research capacity and global perspective of participants; and 

• Forge long-term relationships between U.S. and foreign S&E researchers. 

1 National Science Board. 2008. International Science and Engineering Partnerships: A Priority for U.S. 
Foreign Policy and Our Nation's Innovation Enterprise. NSB-08-4. National Science Foundation, Arlington, 
VA, p. 1. 

2 National Science Board. 2001. Toward a More Effective Role for the U.S. Government in International 
Science and Engineering. NSB-01-187. National Science Foundation, Arlington, VA. 

3 National Science Board. 2000. Toward a More Effective NSF Role in International Science and Engineering, 
Interim Report. NSB-00-217. National Science Foundation, Arlington, VA. 

4 NSB. 2000, p. 10. 

5 A list and descriptions of NSF postdoctoral opportunities can be found at 
http://www.nsf. gov/funding/education. jsp?fund_type=3 

6 National Science Foundation. 2006a. IRFP Program Solicitation. Retrieved from 
http://www.nsf.gOv/pubs/2006/nsf06582/nsf06582.html#pgm_desc_txt 
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This report summarizes the findings from the evaluation of IRFP, which was designed to investigate 
the characteristics of IRFP applicants and their motivations for participation, the host researchers' 
motivations for participating in the program, participants' experiences in and perceptions of the 
program, and outcomes of the program. The full evaluation report is available online at 
http:/www.abtassociates.com/Reports/2012/Evaluation-of-NSF's-lnternational-Research- 
Fellows. aspx. 7 

The importance of international collaboration cannot be overstated. Researchers and policymakers 
agree that while the United States remains a leader in S&E research and development, the nation 
cannot maintain this position without engaging in international collaborations. Promoting 
international engagement at all levels is crucial to fostering successful research partnerships and 
developing the next generation of S&E researchers. Providing early career researchers with an 
opportunity to engage in an international research experience may help them improve their own 
research capabilities and pursue future collaborations with international colleagues. This, in turn, 
could lead U.S. researchers to reap benefits such as increased visibility in the research community, 
access to more substantial funding and resources, and the opportunity to benefit from the expertise 
of international peers. 


Study Findings 


Findings from the evaluation of IRFP revealed important benefits of the program, on both an 
individual and a collective basis 


Key Findings 


To align with the program goals, the evaluation paid particular attention to the opportunities for 
international research collaborations, the experiences to develop research capacity and global 
perspectives, and the relationships between U.S .and foreign researchers. Specifically, the 
evaluation examined the following questions: 

1. Does the extent to which former fellows engage in international collaborations differ from 
that of unfunded applicants? 

2. Do fellows' post-award career activities and job characteristics differ from those of 
unfunded applicants? 

3. What are the perceived outcomes of program participation? 

4. Do the outcomes of program participation extend beyond the direct participants? 

The evaluation provides evidence that the IRFP experience leads to greater levels of international 
research engagement among fellows. 


Martinez, A., Carter, E., Parsad, A. & Whittaker, K. 2012. Evaluation of NSF's International Research 
Fellowship Program: Final Report. Abt Associates Inc., Cambridge, MA. 
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• There are statistically significant and positive differences between fellows and unfunded 
applicants on the number of international postdoctoral fellowships held. 

• IRFP fellows had a larger number of publications with a foreign co-author compared to non- 
funded applicants, and the percentage of publications with a foreign co-author was also 
greater for fellows. These differences were statistically significant. 

Importantly, this international focus does not come at the expense of research productivity or 
professional advancement. 


• Fellows and their peers were equally likely to hold multiple postdoctoral appointments, and 
were equally productive researchers, equally likely to hold a faculty rank of assistant, 
associate, or full professor, and equally likely to be tenured. 

• Career outcomes of IRFP fellows, and applicants overall, compare well against national 
STEM PhD holders on employment, publications, and international collaborations, 
suggesting that IRFP attracts a talented pool of applicants. 


Former IRFP fellows also reported that they reaped positive career outcomes from their 
participation in IRFP. 


• About four-fifths of fellows (79 percent) reported 
that participating in IRFP had qualified them for a 
broader range of career options, and 68 percent 
said that IRFP had made them more competitive 
for jobs. 

• Fellows reported that IRFP had opened up new 
areas for investigation (71 percent). 


The IRFP was a major 
contributing factor to obtaining 
my tenure-track faculty position. 
It also was a defining period in 
development of my research and 
the experience has provided me 
with a lot of the tools and skills I 


The relationships developed during IRFP seed subsequent professional collaborations and activities 
that endure beyond the fellowship period. 


My career was completely 
changed with that experience. I 
believe that research experience 
was the major factor in every 
career move I have made since 
then. Currently I have numerous 
collaborators from that research 
group, as well, and this continues 
to enrich my life/ professional 
experience (IRFP fellow) 


Nearly three-quarters (71 percent) of IRFP fellows 
reported that they had made important 
connections to researchers in their host country. 

Most fellows reported that the fellowships offer 
opportunities for professional relationships that 
endure beyond the fellowship period, through 
subsequent collaborations with their hosts and/or 
additional communications (46 percent each). Half 
of all former fellows said that participation in IRFP 
made them more committed to international 
research collaboration. 
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Further, the program demonstrates potential to reach beyond its immediate participants. 

• More than three-quarters of former IRFP fellows reported that post-fellowship, they taught 
colleagues, students or peers methods learned during their fellowship (78 percent), and 
shared resources or tools acquired during this time (75 percent). 

• Flosts also served as a mechanism for extending the effects of the program to other 
scientists. Twenty-five percent of former IRFP hosts reported that a collaboration with a U.S. 
researcher resulted from their participation in the IRFP program. 


Additional Findings 


The evaluation was also designed to address the following questions: 

1. What are the characteristics of people who apply for and participate in the IRFP program? 

2. What motivates individuals to apply for and participate in the program, and what are 
individuals' experiences during the application process? 

3. What are the program experiences of program participants and managers? 

Since 1992, IRFP has received 1,660 applications from individuals, with varied backgrounds, 
interested in pursuing their research in settings abroad. 

• IRFP is a fairly selective program, accepting just over one-third (35 percent) of applications 
overall since its inception, and it has become more competitive overtime. 

• Nearly half of the proposed research projects 
were proposed in the life sciences (47 
percent), and about one-quarter in the 
physical sciences (26 percent). 

• IRFP applicants proposed research in regions 
that spanned the globe, although a majority 
(60 percent) identified locations in Europe. 

• Graduate advisors are reported to be 
supportive of their students' applications; 72 
percent of applicants reported their advisors 
actively encouraged or supported their 
decision to apply. 

The IRFP applicants represent early career scientists with varied motivations for participating in the 
program, although the motivations reflect the value that these scientists and engineers place on 
foreign collaborations. 

• A large majority of applicants cited the desire to conduct research with a specific person or 
at a specific institution (87 percent), and to enhance their skills or knowledge as a 
researcher (82 percent) as the reasons for applying to IRFP. Over half also noted their desire 


[The IRFP fellowship] allowed me 
to [do] research that was possible 
only by working in collaboration 
with the host institution and 
conducting fieldwork in the host 
country. I chose this country 
specifically because regional 
biogeography allowed me to follow 
the path of inquiry that I had 
hoped to examine. (IRFP applicant) 
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to collaborate with a foreign scientist in general (60 percent) and to enhance their resume 
(57 percent). 

• In selecting a specific host country, applicants were primarily motivated by the presence of a 
host who was conducting research relevant to their own interests (87 percent), followed by 
resources at the host institution that were helpful in their research (52 percent). 

Funded fellows are hosted by research scientists in foreign institutions, who also express a variety of 
motives for participating in the program. Specifically, 

• Many foreign hosts had prior academic and professional experiences in the U.S., although 
66 percent had never hosted a postdoctoral fellow from the U.S. 

• The large majority of foreign host scientists (83 percent) reported that they agreed to host 
fellows because of the specific research projects proposed; the next most commonly 
reported rationale was an interest in creating an international environment in their research 
group (60 percent). 

Overwhelmingly, IRFP participants were satisfied with the program and found the fellowship 
valuable. 

• Former fellows and hosts offer strong 
endorsement for the IRFP program. All former 
IRFP fellows would recommend the program to a 
colleague, and the large majority of hosts (84 
percent) would recommend, or have already 
recommended, to others that they host an IRFP 
postdoctoral fellow from the U.S. 

• Over 90 percent of fellows were somewhat or 
very satisfied with the quality of research they 
were able to conduct as part of IRFP (92 percent) 
and the timing of the fellowship with respect to 
their career goals (97 percent). 

• Over 80 percent of fellows were satisfied with many aspects of their host, including the 
match between research interests, the host's expertise and contributions to the research, 
their inclusion in collaborations, and frequency of meetings. 

• Flosts noted that IRFP fellows compared favorably to other postdoctoral fellows with whom 
hosts had worked; 56 percent of hosts were much or somewhat more satisfied with IRFP 
fellows compared to other postdoctoral fellows, and another 28 percent were equally 
satisfied. 

• Flosts generally strongly agreed or agreed (72 and 24 percent, respectively) that their IRFP 
fellow had sufficient knowledge and expertise, and that the fellow integrated well with their 
research group (66 and 26 percent, respectively). 


[The fellow] led a large 
experimental study still ongoing 
that represented one of the most 
ambitious and interesting 
experiments ever conducted in 
my lab. This study also brought in 
other collaborators (students and 
Pis) and represents one of the 
most satisfying collaborations of 
my career. (IRFP host) 
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The fellowships provided opportunities for collaborations and advances in research. A majority of 
fellows credited their IRFP fellowship with allowing them to make substantial advancements in their 
research, and hosts and fellows agreed they worked collaboratively on important aspects of 
research projects. 

• About two-thirds commented that their IRFP fellowship provided the chance to familiarize 
themselves with the scientific enterprise in their host site (65 percent) and to make 
substantial advancements in their research (64 percent). 

• Hosts and fellows most commonly cited collaborating on activities that involved developing 
project ideas and hypotheses, interpreting results, and planning follow-up work. 

• Hosts and fellows agreed that fellows worked mostly independently collecting data, carrying 
out simulations, and keeping records or tracking supplies and resources. 

IRFP also offers fellows a broad range of professional opportunities beyond the chance to conduct 
quality research at their host institution, including opportunities to network with colleagues from 
other than their host institution, attend lectures in their field, and give talks at their host institution. 

• Other professional activities featured prominently in the experiences of many fellows. 
Specifically, IRFP provided fellows with the opportunity to network with colleagues from 
institutions outside their host institution (90 percent), attend lectures in their field (89 
percent), visit other institutions (89 percent), attend conferences (84 percent), and give 
talks at their institution (71 percent). 

• Fellows also experienced a variety of cultural activities including sightseeing (96 percent) 
outdoor activities that explore geography (95 percent), museums (89 percent), and festivals 
or holidays (85 percent). 

Most participants did not cite specific challenges, but those that were mentioned illustrate areas 
that might deserve attention of future fellows or hosts. 

• Forty percent of fellows reported no noteworthy difficulties during their fellowship. The 
most common challenges were related to logistics (32 percent) and language difficulties (20 
percent), followed by inadequate access to space or resources (16 percent), and not enough 
guidance from the host researcher or research group (15 percent). 

• Only one-third of hosts reported any challenges experienced in hosting an IRFP fellow. 
Specific challenges in hosting an IRFP fellow were identified by less than 10 percent of hosts, 
including noting that the fellow did not devote enough time to the research collaboration 
(10 percent), and the fellow worked too independently or did not work well as a 
collaborator (8 percent). 


Evaluation Approach 


The evaluation employed quasi-experimental impact analyses to compare the outcomes of fellows 
to those of unfunded applicants, using pre-award characteristics of applicants to mitigate the 
potential threat of selection bias. To reduce the risks associated with selection bias, the study 
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incorporated propensity score analysis (PSA) to construct groups of awardees and non-awardees 
that were statistically similar across a number of pre-existing characteristics. A secondary set of 
comparative analyses between IRFP applicants (and fellows) and a nationally representative sample 
of STEM doctorates from the Survey of Doctoral Recipients (SDR) was used to situate the outcomes 
of IRFP program participants and applicants within the national S&E context. The evaluation also 
used descriptive analyses to explore the pre-award international research experiences and other 
characteristics of IRFP applicants and host scientists; to understand what motivated STEM doctoral 
recipients to apply for an IRFP postdoc and what led IRFP host scientists to collaborate with an IRFP 
fellow; and to describe the experiences of IRFP participants (i.e., fellows and hosts), both during and 
after the period of the postdoctoral fellowship. 

Data for the evaluation were drawn from extant sources, and surveys were administered to collect 
information from program applicants (both those who received IRFP fellowships and those who did 
not) and IRFP foreign hosts. Extant data came from NSF's administrative records on applicants and 
from the Survey of Doctoral Recipients (SDR). The core data for the evaluation were gathered 
through online surveys completed— from January through March, 2011— by IRFP applicants and 
hosts. 

The target populations for the study included all individuals who had applied to the IRFP program 
from its inception in 1992 through 2009, as well as the research scientists who served as foreign 
hosts during this period. Responses were received from 457 IRFP fellows and 582 unfunded 
applicants; responses rates were 81 percent and 55 percent, respectively, and estimates were 
adjusted for non-response. Surveys were received from 328 hosts, for a 61 percent response rate. 


Summary 


Findings from this evaluation provide ample evidence that IRFP fulfills its promise of providing 
opportunities for collaborations among U.S. and foreign researchers in settings abroad. Unique 
among NSF postdoctoral programs, IRFP provides early career post-PhD scientists the opportunity to 
engage in overseas scientific research and collaborations with scientists outside the U.S. This study 
of IRFP examined the characteristics and motivations of IRFP participants, the opportunities for 
research and professional growth provided through IRFP, and the professional outcomes related to 
the program. 

The evaluation found evidence that the IRFP program is meeting its goals to: introduce early career 
scientists and engineers to opportunities for international research collaboration; build research 
capacity and global perspective of participants; and forge long-term relationships between U.S. and 
foreign S&E researchers. The contributions of the IRFP program in each of these areas are 
summarized below. 


Opportunities for International Research Collaboration 


IRFP is a selective program that offers international research opportunities to interested individuals. 
Both unfunded applicants and successful fellows reported being attracted to the program because it 
offered them an opportunity to conduct research with individuals in foreign settings and at 
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institutions outside the U.S. Foreign hosts reported that the proposed research topics were 
engaging, as was the potential to create international collaborations in their research group. 

IRFP attracts a talented pool of applicants from among the pool of national STEM PhD holders, who 
reported interest in the contributions that opportunities abroad may make in their own research. 
After their experiences, half of all former fellows said that participation in IRFP made them more 
committed to international research collaboration. 

IRFP fellows indicated that they engaged in research collaborations while abroad, and an 
overwhelming majority of fellows were satisfied with the quality of research conducted as part of 
IRFP and the timing of the fellowship with respect to their career goals. Flosts and fellows agreed 
they worked collaboratively on developing project ideas and hypotheses, interpreting results, and 
planning follow-up work. A majority of fellows also credited their IRFP fellowship with allowing them 
to make substantial advancements in their research. 

Further, IRFP offers fellows a broad range of professional opportunities beyond the opportunity to 
conduct quality research at their host institution, including opportunities to network with colleagues 
from other than their host institutions, attend lectures in their field, attend conferences, and give 
talks at their host institution. 

Flosts provided positive comments about their experiences with IRFP and the fellows specifically. A 
majority agreed that their IRFP fellow had sufficient knowledge and expertise for the research 
conducted and that the fellow integrated well with their research group. Further, hosts noted that 
IRFP fellows compared favorably to other postdoctoral fellows with whom hosts had worked. 


Research Capacity and Global Perspectives 


A unique feature of IRFP is its focus on providing international research experiences for postdoctoral 
fellows. Individuals who participate in IRFP develop their knowledge and research skills, broaden 
their perspectives, and contribute to the global understanding and research enterprise more 
broadly. A majority of fellows credited IRFP with opening up new areas for investigation and 
familiarizing them with the scientific enterprise in their host site. Three-quarters of fellows also 
observed that IRFP qualified them for a broader range of career options, and two-thirds felt it 
contributed to making them more competitive for jobs. 

The evaluation provides evidence that these experiences lead to greater levels of international 
research engagement among fellows. Fellows ranked consistently higher than unfunded applicants 
on several facets of international research, including the number of international postdoctoral 
fellowships, the number of publications with a foreign co-author, and the percentage of publications 
with a foreign co-author. The fellows' research productivity is consistently higher across all three 
measures. 

Importantly, this international focus did not come at the expense of research productivity or career 
advancement. Specifically, fellows and their peers were equally likely to hold multiple postdoctoral 
appointments, and were equally productive researchers, equally likely to hold a faculty rank of 
assistant, associate, or full professor, and equally likely to be tenured. 
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Further, the program demonstrates potential to reach beyond the immediate participants. More 
than three-quarters of former IRFP fellows shared resources or tools acquired during their 
international postdoctoral fellowship and taught colleagues, students or peers methods learned 
during this time. 


Long-Term Relationships between U.S. and Foreign Researchers 


The relationships developed during IRFP seed subsequent professional collaborations and activities. 
Many IRFP fellows developed professional relationships that endured beyond the fellowship period, 
through subsequent collaborations with their hosts and/or additional communications. The 
continuing collaborations reported by a large proportion of former fellows and hosts included co- 
authored papers and exchanged ideas, data, results or tools. They also visited each other at their 
respective institutions, and co-advised students. Also, there is evidence that the collaborations may 
be seeding networks, as fellows interact with hosts' research groups, and as hosts establish 
collaborations with other U.S researchers as a result of their participation in the IRFP program. 


Conclusion 


The evaluation demonstrated that participants 
derive important benefits from the program, 
on an both individual and a collective basis. 
Former fellows and hosts strongly endorsed 
the IRFP program, and would recommend IRFP 
to their students and colleagues. IRFP offers 
an opportunity for interested researchers to 
develop their international research 
collaborations early in their careers. The 
experiences during IRFP seed relationships 
that often are sustained and that generate 
international research collaborations across 
geopolitical boundaries. As such, IRFP has a 
central role in NSF's efforts to respond to the 
NSB's charge that NSF play a leadership role in 
international S&E research and education 
activities, and specifically in promoting 
international S&E among early career 
scientists and engineers. 


This is a great program. It helped launch 
my career. I didn't know the quality of 
people I would be working with, but really 
got very lucky. They took me under their 
wing, and I met many researchers at 
several different institutions over the 
course of the fellowship. The country 
benefits greatly from a program like this. 
During the 13 years since completing the 
fellowship, I've been working on various ... 
research projects as a principal 
investigator. The small amount of funds 
that the U.S. government invested in me 
during the IRFP I believe enabled my 
subsequent research contributions to 
these government agencies and goals.... 
Keep funding this program. It has a great 
deal of payback. (IRFP fellow) 


Abt Associates Inc. 


10 






